2 Fundamentals of Information, Signal and System

Theory

In biosignal processing, a variety of methods from different disciplines are applied

to problems in medicine. The methods of classical information and signal processing

form an important basis. The following chapter introduces methods from these discip-

lines that are commonly used in biosignal processing and discusses the context along

practical examples.

2.1 Information and Information Transmission

The concept of information occupies a crucial place in all scientific disciplines, in-

cluding biosignal processing. However, the concept of information is not uniformly

defined in the various disciplines and has different meanings. The most common is

the concept of information coined by C. E. Shannon¹, which is used in communica-

tions engineering.

Digital Information

However, the Shannonian definition of digital information refers exclusively to statist-

ical aspects of information, i.e. the probability of occurrence of characters of a char-

acter set in a string to be transmitted. The statistical information content I of this se-

quence of n characters with the respective occurrence probability pi of the individual

characters is:

I =

n

∑

i=1

log2

1

pi

= log2

1

p1

+ log2

1

p2

+ ⋅⋅⋅+ log2

1

pn

≥0 ,

n ∈ℤ.

(2.1)

According to this equation, the information increases steadily with the number n of

characters in the chain, as long as the following relation holds for a character from a

character set with N different characters:

0 ≤pj ≤1 ,

N

∑

j=1

pj = 1 .

(2.2)

Equation 2.2 stands for the existence of a character and at the same time guarantees

that each character appears only once in the character set. From the two equations

it becomes clear that the information content of a character string increases strongly

1 Claude Elwood Shannon (1916–2001), US mathematician and electrical engineer, founder of inform-

ation theory

https://doi.org/10.1515/9783110736298-002